Information processing apparatus and information processing method to calibrate line-of-sight of a user

ABSTRACT

An information processing apparatus includes: a line-of-sight information acquisition unit that acquires line-of-sight information of a user and a calibration execution unit that executes calibration on the basis of a position of a target object in a visual field and the line-of-sight information, the target object has a visual attraction degree higher than a first threshold and a size in a predetermined direction smaller than a predetermined size, in a case where the target object has been detected from the visual field of the user.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International PatentApplication No. PCT/JP2017/047366 filed on Dec. 28, 2017, which claimspriority benefit of Japanese Patent Application No. JP 2017-060995 filedin the Japan Patent Office on Mar. 27, 2017. Each of theabove-referenced applications is hereby incorporated herein by referencein its entirety.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an information processing method, and a program.

BACKGROUND ART

In recent years, a technology of executing processing according to theline-of-sight of a user has been developed. However, an eye structure isusually different among users. For example, the users usually havedifferent sizes of eyeballs. Furthermore, the positional relationshipbetween the eyes of the user and a device can vary depending ondifferences in the devices used by the users. Therefore, there is apossibility that an error occurs in the detection of the line-of-sightof the user, and a technology for improving the accuracy of thedetection of the line-of-sight of the user has also been developed.

For example, a technology of performing calibration of the line-of-sightbefore the detection of the line-of-sight of the user has beendeveloped. For example, in the technology described in Patent Document 1below, the calibration is executed on the basis of the line-of-sight ofa user and a set position of an image utilized to determine whether ornot to unlock. Moreover, in the technology described in Patent Document2 below, the calibration is executed on the basis of a position of anobject manipulatable by a user and the line-of-sight of the user at thetime of manipulating the object.

CITATION LIST Patent Document

Patent Document 1: Japanese Patent Application Laid-Open No. 2015-153302

Patent Document 2: International Publication No. 2016/139850

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

However, it is desirable that the situation in which the calibration ofthe line-of-sight can be executed be not very limited. Accordingly, itis desirable to provide a technology capable of enabling the calibrationof the line-of-sight to be executed more naturally.

Solutions to Problems

According to the present disclosure, provided is an informationprocessing apparatus including: a line-of-sight information acquisitionunit configured to acquire line-of-sight information of a user; and acalibration execution unit configured to execute calibration on thebasis of a position of a target object in a visual field and theline-of-sight information, the target object having a visual attractiondegree higher than a first threshold and a size in a predetermineddirection smaller than a predetermined size, in a case where the targetobject has been detected from the visual field of the user.

According to the present disclosure, provided is an informationprocessing method including: acquiring line-of-sight information of auser; and executing calibration by a processor on the basis of aposition of a target object in a visual field and the line-of-sightinformation, the target object having a visual attraction degree higherthan a first threshold and a size in a predetermined direction smallerthan a predetermined size, in a case where the target object has beendetected from the visual field of the user.

According to the present disclosure, provided is a program for causing acomputer to function as an information processing apparatus, theinformation processing apparatus including:

a line-of-sight information acquisition unit configured to acquireline-of-sight information of a user; and a calibration execution unitconfigured to execute calibration on the basis of a position of a targetobject in a visual field and the line-of-sight information, the targetobject having a visual attraction degree higher than a first thresholdand a size in a predetermined direction smaller than a predeterminedsize, in a case where the target object has been detected from thevisual field of the user.

Effects of the Invention

As described above, according to the present disclosure, provided is atechnology capable of enabling the calibration of the line-of-sight tobe executed more naturally. Note that the above-described effects arenot necessarily limited, and any one of the effects shown in thisspecification or other effects grasped from this specification may beexerted together with the above-described effects or instead of theabove-described effects.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a configuration example of an informationprocessing system according to a first embodiment.

FIG. 2 is a diagram showing a functional configuration example of theinformation processing apparatus.

FIG. 3 is a diagram showing a detailed configuration example of thecontrol unit.

FIG. 4 is a diagram showing an example of the visual field of the user.

FIG. 5 is a diagram for describing a case of determining that thecalibration is not to be executed on the basis of the line-of-sightmotion prediction information.

FIG. 6 is a diagram for describing a case of determining that thecalibration is to be executed on the basis of the line-of-sight motionprediction information.

FIG. 7 is a diagram showing an example of displayed informationindicating the completion of the calibration execution.

FIG. 8 is a flowchart showing an operation example of the calibration.

FIG. 9 is a diagram showing a first example of recalibration execution.

FIG. 10 is a diagram showing a second example of the recalibrationexecution.

FIG. 11 is a diagram for describing an example of disabling thecalibration.

FIG. 12 is a diagram showing a third example of the recalibrationexecution.

FIG. 13 is a flowchart showing an operation example of therecalibration.

FIG. 14 is a diagram for describing a first example of a techniqueaccording to a modification example.

FIG. 15 is a diagram for describing a second example of the techniqueaccording to the modification example.

FIG. 16 is a diagram for describing a third example of the techniqueaccording to the modification example.

FIG. 17 is a block diagram showing a hardware configuration example ofthe information processing apparatus.

MODE FOR CARRYING OUT THE INVENTION

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings. Notethat, in this specification and the drawings, constituent elementshaving substantially the same functional configurations are denoted bythe same reference signs to omit redundant description.

Moreover, in this specification and the drawings, there may be a casewhere a plurality of constituent elements having substantially the sameor similar functional configurations is distinguished by addingdifferent figures after the same reference signs. However, in a casewhere it is not necessary to particularly distinguish each of theplurality of constituent elements having substantially the same orsimilar functional configurations, only the same reference signs areused. Furthermore, there may be case where similar constituent elementsin different embodiments are distinguished by adding different alphabetsafter the same reference signs. However, in a case where it is notnecessary to particularly distinguish each of the similar constituentelements, only the same reference signs are used.

Note that the description will be given in the following order.

0. Overview

1. Details of Embodiments

1.1 System Configuration Example

1.2. Functional Configuration Example of Information ProcessingApparatus

1.3. Function Details of Information Processing System

1.3.1. Details of Calibration

1.3.2. Recalibration

1.3.3. Various Modification Examples

2. Hardware Configuration Example

3. Conclusion

0. Overview

First, the overview of the embodiments of the present disclosure will bedescribed. In recent years, a technology of executing processingaccording to the line-of-sight of a user has been developed. However, aneye structure is usually different among users. For example, the usersusually have different sizes of eyeballs. Furthermore, the positionalrelationship between the eyes of the user and a device can varydepending on differences in the devices used by the users. Therefore,there is a possibility that an error occurs in the detection of theline-of-sight of the user, and a technology for improving the accuracyof the detection of the line-of-sight of the user has also beendeveloped.

For example, a technology of performing calibration of the line-of-sightbefore the detection of the line-of-sight of the user has beendeveloped. For example, there is a technology of executing thecalibration on the basis of the line-of-sight of the user and a setposition of an image utilized to determine whether or not to unlock.Furthermore, there is a technology of executing the calibration on thebasis of a position of an object manipulatable by a user and theline-of-sight of the user at the time of manipulating the object.

However, it is desirable that the situation in which the calibration ofthe line-of-sight can be executed be not very limited. For example, ifthe calibration of the line-of-sight cannot be executed unless aspecific image is displayed (or unless a manipulatable object exists inthe visual field of the user), the situation in which the calibration ofthe line-of-sight can be executed is limited. Accordingly, in thisspecification, a technology capable of enabling the calibration of theline-of-sight to be executed more naturally will be mainly described.

The overview of the embodiments of the present disclosure has beendescribed above.

1. Details of Embodiments

First, the details of the embodiments of the present disclosure will bedescribed.

1.1. System Configuration Example

First, a configuration example of an information processing systemaccording to the embodiments of the present disclosure will be describedwith reference to the drawings. FIG. 1 is a diagram showing aconfiguration example of the information processing system according tothe embodiments of the present disclosure. As shown in FIG. 1, theinformation processing system according to the embodiments of thepresent disclosure includes an information processing apparatus 10. Theinformation processing apparatus 10 is used by a user U1.

Note that a case where the information processing apparatus 10 is a headmounted display (HMD) worn on the head of the user U1 will be mainlydescribed in the present embodiment. In particular, in the presentembodiment, a case where the information processing apparatus 10 is asee-through HMD will be mainly described. However, the informationprocessing apparatus 10 is not limited to the HMD. For example, theinformation processing apparatus 10 may be a smartphone or may be atablet terminal.

In the example shown in FIG. 1, the user U1 wearing the informationprocessing apparatus 10 on the head is looking out of a window M1 from aroom. A real space R1 exists outside the window M1, and the user U1 canvisually recognize the real space R1 through the information processingapparatus 10 (the visual field of the user U1 includes the real spaceR1). Herein, any object may exist in the real space R1. In the presentembodiment, the line-of-sight of the user U1 to an object existing inthe real space R1 is utilized for the calibration. Hereinafter, mainlypresumed is a case where buildings B1 to B3 exist as an example of oneor a plurality of objects in the real space R1.

The configuration example of the information processing system accordingto the embodiments of the present disclosure has been described above.

1.2. Functional Configuration Example of Information ProcessingApparatus

Next, a functional configuration example of the information processingapparatus 10 will be described. FIG. 2 is a diagram showing thefunctional configuration example of the information processing apparatus10. As shown in FIG. 2, the information processing apparatus 10 has avisual field analysis imaging unit iii, a line-of-sight detectionimaging unit 112, a sensor unit 113, a control unit 120, a storage unit130, a communication unit 140 and a display unit 150. Furthermore, theinformation processing apparatus 10 may be connected to a serverapparatus (not shown) via a communication network. The communicationnetwork includes, for example, the Internet.

The visual field analysis imaging unit 111 has a function of obtainingan image (visual field analysis image) by imaging the visual field ofthe user U1. For example, the visual field analysis imaging unit 111includes a camera (including an image sensor) and obtains the visualfield analysis image captured by the camera. The number of camerasincluded in the visual field analysis imaging unit 111 is notparticularly limited as long as the number is one or more. Then, theposition at which the visual field analysis imaging unit 111 is providedis not particularly limited either. For example, the visual fieldanalysis imaging unit 111 may be integrated with the informationprocessing apparatus 10 or may exist as an entity separated from theinformation processing apparatus 10.

The line-of-sight detection imaging unit 112 has a function of obtainingan image (line-of-sight detection image) by imaging the eyes of the userU1. For example, the line-of-sight detection imaging unit 112 includes acamera (including an image sensor) and obtains the line-of-sightdetection image captured by the camera. The number of cameras includedin the line-of-sight detection imaging unit 112 is not particularlylimited as long as the number is one or more. Then, the position atwhich the line-of-sight detection imaging unit 112 is provided is notparticularly limited either. For example, the line-of-sight detectionimaging unit 112 may be integrated with the information processingapparatus 10 or may exist as an entity separated from the informationprocessing apparatus 10.

The sensor unit 113 is constituted by including a sensor and has afunction of detecting the visual field motion of the user U1. Forexample, the sensor unit 113 may be constituted by including anacceleration sensor and detect the visual field motion of the user U1 bythe acceleration detected by the acceleration sensor. Alternatively, thesensor unit 113 may be constituted by including a gyro sensor and detectthe visual field motion of the user U1 by the angular velocity detectedby the gyro sensor. Note that the sensor unit 113 does not have to beprovided in a case where the visual field motion of the user U1 isdetected on the basis of the visual field analysis image captured by thevisual field analysis imaging unit 111.

The communication unit 140 is constituted by including a communicationcircuit and has functions of acquiring data from a server apparatus (notshown) connected to the communication network and providing data to theserver apparatus (not shown) via the communication network. For example,the communication unit 140 is constituted by a communication interface.Note that the number of server apparatuses (not shown) connected to thecommunication network may be one or may be plural.

The storage unit 130 is constituted by including a memory and is arecording medium that stores a program executed by the control unit 120and stores data necessary for the execution of the program. Furthermore,the storage unit 130 temporarily stores data for arithmetic operationsby the control unit 120. The storage unit 130 is constituted by amagnetic storage unit device, a semiconductor storage device, an opticalstorage device, a magneto-optical storage device, or the like.

The display unit 150 has a function of displaying various screens. Thetype of the display unit 150 is not limited. For example, the displayunit 150 only needs to be a display capable of performing visuallyrecognizable display to the user and may be a liquid crystal display ormay be an organic electro-luminescence (EL) display.

The control unit 120 executes control of each unit of the informationprocessing apparatus 10. FIG. 3 is a diagram showing a detailedconfiguration example of the control unit 120. As shown in FIG. 3, thecontrol unit 120 includes a motion acquisition unit 121, an objectrecognition unit 122, a visual attraction degree calculation unit 123, atarget object detection unit 124, a prediction information generationunit 125, a gesture recognition unit 126, a line-of-sight informationacquisition unit 127, a calibration execution unit 128 and a displaycontrol unit 129. The detail of each of these functional blocks will bedescribed later. Note that the control unit 120 may be constituted by,for example, one or a plurality of central processing units (CPU). In acase where the control unit 120 is constituted by a processing unit suchas a CPU, the processing apparatus may be constituted by an electroniccircuit.

The functional configuration example of the information processingapparatus 10 according to the present embodiment has been describedabove.

1.3. Function Details of Information Processing System

Next, the function details of the information processing systemaccording to the present embodiment will be described.

1.3.1. Details of Calibration

FIG. 4 is a diagram showing an example of the visual field of the user.Note that the information processing apparatus 10 worn on the head ofthe user is omitted in FIG. 4. As shown in FIG. 4, in the visual fieldof the user, the buildings B1 to B3 exist as an example of one or aplurality of objects existing in the real space R1.

Herein, among the one or plurality of objects existing in the visualfield of the user U1, it is considered that the line-of-sight of theuser U1 is highly likely to meet an object with a visual attractiondegree higher than a certain threshold (first threshold). In the presentembodiment, the calibration is executed by utilizing the fact that theline-of-sight of the user U1 is highly likely to meet the object with avisual attraction degree higher than the first threshold. In the exampleshown in FIG. 4, suppose that the visual attraction degrees of thebuildings B1 to B3 are higher than the first threshold.

Moreover, among the objects with visual attraction degrees higher thanthe first threshold, it is considered that an error, in a predetermineddirection, of the position that the line-of-sight meets is smaller andthe accuracy of the calibration is improved, in a case where theline-of-sight meets an object with a size in the predetermined directionsmaller than a predetermined size as compared to a case where theline-of-sight meets an object with a size in the predetermined directionlarger than the predetermined size. Accordingly, in the presentembodiment, the calibration is executed by utilizing the line-of-sightof the user U1 toward an object with a size in the predetermineddirection smaller than the predetermined size.

Note that the predetermined direction may be a horizontal direction, maybe a vertical direction or may be another direction. Referring to FIG.4, the sizes of the buildings B1 to B3 in the horizontal direction aredenoted by W1 to W3, respectively. For example, in the example shown inFIG. 4, suppose that the size W1 of the building B1 in the horizontaldirection is larger than the predetermined size. On the other hand,suppose that the size W2 of the building B2 in the horizontal directionis smaller than the predetermined size, and the size W3 of the buildingB3 in the horizontal direction is smaller than the predetermined size.

Hereinafter, the details of the calibration according to the presentembodiment will be described. Herein, the timing when the calibration isexecuted is not limited. For example, the calibration may be executed atthe timing when the information processing apparatus 10 is activated,may be executed at predetermined time intervals, or may be executed in acase where the user has manipulated for the instruction that thecalibration is to be executed. However, it is considered that theaccuracy of the calibration is higher if the calibration is executed ina case where the visual field motion of the user is less.

Accordingly, in the present embodiment, the motion acquisition unit 121acquires the visual field motion of the user. For example, as describedabove, the motion acquisition unit 121 acquires the motion detected bythe sensor unit 113. Then, the object recognition unit 122 determineswhether or not the visual field motion of the user acquired by themotion acquisition unit 121 is less than predetermined motion. Theobject recognition unit 122 recognizes one or a plurality of objectsfrom the visual field of the user in a case where the visual fieldmotion of the user acquired by the motion acquisition unit 121 is lessthan the predetermined motion.

Specifically, in a case where the visual field motion of the useracquired by the motion acquisition unit 121 is less than thepredetermined motion, the object recognition unit 122 recognizes one ora plurality of objects from the visual field analysis image captured bythe visual field analysis imaging unit 111. Note that the motionacquisition unit 121 does not have to acquire the visual field motion ofthe user, and the object recognition unit 122 does not have to determinewhether or not the visual field motion of the user is less than thepredetermined motion. Herein, presumed is a case where the objectrecognition unit 122 recognizes a group of objects (including thebuildings B1 to B3) existing in the visual field of the user.

On the basis of the visual field analysis image captured by the visualfield analysis imaging unit 111, the visual attraction degreecalculation unit 123 calculates the visual attraction degrees of thegroup of the objects (including the buildings B1 to B3) recognized bythe object recognition unit 122. Herein, the visual attraction degreemay be a numerical value indicating how much the visual attention of theuser U1 is paid. How the visual attraction degree is specificallycalculated is not limited. For example, the technique of calculating thevisual attraction degree disclosed in Japanese Patent ApplicationLaid-Open No. 2014-170396 may be applied to the technique of calculatingthe visual attraction degree by the visual attraction degree calculationunit 123 according to the present embodiment.

The target object detection unit 124 detects, from the visual field ofthe user, a target object which has a visual attraction degree higherthan the first threshold and a size in the predetermined directionsmaller than the predetermined size. More specifically, the targetobject detection unit 124 detects, from the group of the objects(including the buildings B1 to B3) recognized by the object recognitionunit 122, a target object which has a visual attraction degree higherthan the first threshold and a size in the predetermined directionsmaller than the predetermined size. At this time, all objects, whichhave visual attraction degrees higher than the first threshold and sizesin the predetermined direction smaller than the predetermined size, maybe detected as the target objects. However, some of the objects, whichhave visual attraction degrees higher than the first threshold and sizesin the predetermined direction smaller than the predetermined size, donot have to be detected as the target objects.

For example, also presumed is a case where, among the group of theobjects recognized by the object recognition unit 122, another object,which has a visual attraction degree higher than a second threshold, islocated at a distance from one object shorter than a predetermineddistance. In such a case, it is considered that determining whether theobject that the line-of-sight of the user meets is the one object oranother object is difficult and the accuracy of the calibration is notimproved. Accordingly, in a case where another object, which has avisual attraction degree higher than the second threshold, is located ata distance from the one object shorter than the predetermined distance,the target object detection unit 124 is only required not to detect theone object as the target object.

In the example shown in FIG. 4, the target object detection unit 124detects the buildings B1 to B3 as objects having visual attractiondegrees higher than the first threshold. Furthermore, in the exampleshown in FIG. 4, presumed is a case where the predetermined direction isthe horizontal direction, and the target object detection unit 124detects, from the buildings B1 to B3 having visual attraction degreeshigher than the first threshold, the building B2 and the building B3 asthe objects with the sizes in the horizontal direction smaller than thepredetermined size.

However, in the example shown in FIG. 4, presumed is a case where thetarget object detection unit 124 determines that the building B1 has avisual attraction degree higher than the second threshold and that thebuilding B1 is located at a distance from the building B2 shorter thanthe predetermined distance, and does not detect the building B2 as thetarget object (presumed is a case where the building B3 is detected asthe target object).

The line-of-sight information acquisition unit 127 acquires the userline-of-sight information. More specifically, the line-of-sightinformation acquisition unit 127 acquires the line-of-sight informationindicating the line-of-sight of the user by analyzing the line-of-sightdetection image detected by the line-of-sight detection imaging unit112. In a case where the target object has been detected by the targetobject detection unit 124, the calibration execution unit 128 executesthe calibration on the basis of the position of the target object in thevisual field of the user and the line-of-sight information acquired bythe line-of-sight information acquisition unit 127. This makes itpossible to enable the calibration of the line-of-sight to be executedmore naturally.

At this time, in a case where a target object with a size in thepredetermined direction smaller than the predetermined size has beendetected, an error in the predetermined direction of the position thatthe line-of-sight meets is considered to be small, and the accuracy ofthe calibration in the predetermined direction is considered to beimproved. Accordingly, in such a case, the calibration execution unit128 is only required to execute the calibration for a directionsubstantially parallel to the predetermined direction (e.g., for thepredetermined direction).

For example, in a case where a target object with a size in thehorizontal direction smaller than the predetermined size has beendetected, the calibration execution unit 128 may execute the calibrationfor a direction substantially parallel to the horizontal direction(e.g., for the horizontal direction). Alternatively, in a case where atarget object with a size in the vertical direction smaller than thepredetermined size has been detected, the calibration execution unit 128may execute the calibration for a direction substantially parallel tothe vertical direction (e.g., for the vertical direction). However, thecalibration execution unit 128 may execute the calibration regardless ofthe predetermined direction.

Note that, in a case where a target object with a size in the horizontaldirection smaller than the predetermined size has been detected, thecalibration execution unit 128 may execute the calibration without anyparticular condition, but may also execute the calibration in a casewhere a predetermined condition is met. For example, in a case where atarget object with a size in the horizontal direction smaller than thepredetermined size has been detected, the calibration execution unit 128may control whether or not to execute the calibration on the basis ofline-of-sight motion prediction information of the user generated by theprediction information generation unit 125. Such an example will bedescribed with reference to FIGS. 5 and 6.

FIG. 5 is a diagram for describing a case of determining that thecalibration is not to be executed on the basis of the line-of-sightmotion prediction information. Prior to determining whether or not toexecute the calibration, the prediction information generation unit 125generates the line-of-sight motion prediction information of the user onthe basis of the visual attraction degrees of the group of the objectscalculated by the visual attraction degree calculation unit 123. Theline-of-sight motion prediction information of the user is informationindicating the result of predicting order of the line-of-sight motion ofthe user on the basis of the visual attraction degrees.

First, the prediction information generation unit 125 detects, from thevisual field of the user, a plurality of objects having visualattraction degrees higher than a third threshold. Specifically, on thebasis of the visual attraction degrees of the group of the objectscalculated by the visual attraction degree calculation unit 123, theprediction information generation unit 125 detects a plurality ofobjects having visual attraction degrees higher than the thirdthreshold. Then, the prediction information generation unit 125generates, as the line-of-sight motion prediction information,information in which the respective positions of the plurality ofdetected objects are arranged in descending order of the visualattraction degrees.

In the example shown in FIG. 5, suppose that the buildings B1 to B3 havevisual attraction degrees higher than the third threshold, and among thebuildings B1 to B3, the visual attraction degree of the building B1 isthe highest, the visual attraction degree of the building B2 is thesecond highest, and the visual attraction degree of the building B3 isthe lowest. At this time, the prediction information generation unit 125determines that the buildings B1 to B3 have visual attraction degreeshigher than the third threshold and the visual attraction degrees are indescending order of the buildings B1, the building B2 and the buildingB3, and generates, as the line-of-sight motion prediction information,the information in which a position P1 of the building B1, a position P2of the building B2, and a position P3 of the building B3 are arranged inthis order.

Then, the calibration execution unit 128 acquires, as actualline-of-sight motion measurement information, the information in whichthe positions that the line-of-sight of the user U1 has actually met arearranged in chronological order. More specifically, the calibrationexecution unit 128 acquires the line-of-sight information acquired inchronological order by the line-of-sight information acquisition unit127 and acquires, as the actual line-of-sight motion measurementinformation, the information in which these pieces of line-of-sightinformation are arranged in the acquisition order. Note that thepositions that the line-of-sight has met may be the same positions in acase where the line-of-sight has been meeting the same positions over apredetermined period of time.

In the example shown in FIG. 5, the positions that the line-of-sight ofthe user U1 meets are not in the descending order of the visualattraction degrees, and the line-of-sight of the user U1 meets aposition Q11, a position Q12 and a position Q13 in chronological order.In other words, the line-of-sight information indicating each of theposition Q11, the position Q12 and the position Q13 is acquired by theline-of-sight information acquisition unit 127 in chronological order,and the calibration execution unit 128 acquires, as the actualline-of-sight motion measurement information, the information in whichthese pieces of line-of-sight information are arranged in theacquisition order.

Then, in a case where a matching degree between the line-of-sight motionprediction information and the actual line-of-sight motion measurementinformation does not exceed a predetermined reference value, it isconsidered that the user U1 moves the line-of-sight on the basis of acriterion different from the visual attraction degree. Thus, thecalibration execution unit 128 does not have to execute the calibrationin this case. Herein, the matching degree may be calculated in any way.As one example, the calibration execution unit 128 may calculate thematching degree between the line-of-sight motion prediction informationand the actual line-of-sight motion measurement information to besmaller as the sum (or the average value) of the distances (or thesquares of the distances) between the positions with correspondingorders in the line-of-sight motion prediction information and the actualline-of-sight motion measurement information is greater.

For example, in the example shown in FIG. 5, in the line-of-sight motionprediction information and the actual line-of-sight motion measurementinformation, the orders of the positions P1 and Q11 correspond to eachother, the orders of the positions P2 and Q12 correspond to each other,and the orders of the positions P3 and Q13 correspond to each other.Therefore, the calibration execution unit 128 may calculate the matchingdegree between the line-of-sight motion prediction information and theactual line-of-sight motion measurement information on the basis of theaverage value of the squares of the distance between the positions P1and Q11, the distance between the positions P2 and Q12 and the distancebetween the positions P3 and Q13. In the example shown in FIG. 5,presumed is a case where the calibration execution unit 128 does notexecute the calibration because the matching degree does not exceed thepredetermined reference value.

FIG. 6 is a diagram for describing a case of determining that thecalibration is to be executed on the basis of the line-of-sight motionprediction information. Also in the example shown in FIG. 6, similar tothe example shown in FIG. 5, the prediction information generation unit125 determines that the buildings B1 to B3 have visual attractiondegrees higher than the third threshold and the visual attractiondegrees are in descending order of the buildings B1, the building B2 andthe building B3, and generates, as the line-of-sight motion predictioninformation, the information in which the position P1 of the buildingB1, the position P2 of the building B2, and the position P3 of thebuilding B3 are arranged in this order.

On the other hand, in the example shown in FIG. 6, the positions thatthe line-of-sight of the user U1 meets are not in the descending orderof the visual attraction degrees, and the line-of-sight of the user U1meets a position Q21, a position Q22 and a position Q23 in chronologicalorder. In other words, the line-of-sight information indicating each ofthe position Q21, the position Q22 and the position Q23 is acquired bythe line-of-sight information acquisition unit 127 in chronologicalorder, and the calibration execution unit 128 acquires, as the actualline-of-sight motion measurement information, the information in whichthese pieces of line-of-sight information are arranged in theacquisition order.

At this time, in a case where the matching degree between theline-of-sight motion prediction information and the actual line-of-sightmotion measurement information exceeds the predetermined referencevalue, it is considered that the user U1 moves the line-of-sight in thedescending order of the visual attraction degrees. Thus, the calibrationexecution unit 128 is only required to execute the calibration in thiscase.

For example, in the example shown in FIG. 6, in the line-of-sight motionprediction information and the actual line-of-sight motion measurementinformation, the orders of the positions P1 and Q21 correspond to eachother, the orders of the positions P2 and Q22 correspond to each other,and the orders of the positions P3 and Q23 correspond to each other.Therefore, the calibration execution unit 128 may calculate the matchingdegree between the line-of-sight motion prediction information and theactual line-of-sight motion measurement information on the basis of theaverage value of the squares of the distance between the positions P1and Q21, the distance between the positions P2 and Q22 and the distancebetween the positions P3 and Q23. In the example shown in FIG. 6,presumed is a case where the calibration execution unit 128 executes thecalibration because the matching degree exceeds the predeterminedreference value. Note that, herein, the matching degree between theline-of-sight motion prediction information and the actual line-of-sightmotion measurement information may be calculated by other techniques.For example, the matching degree between the line-of-sight motionprediction information and the actual line-of-sight motion measurementinformation may be calculated by a matching degree between therespective directions (vectors) of the line-of-sight motion predictioninformation and the actual line-of-sight motion measurement information.

In a case where the calibration has been executed in this manner and theexecution of the calibration has been completed, the user U1 should beaware of the completion of the execution of the calibration.Accordingly, in a case where the calibration execution unit 128 hascompleted the execution of the calibration, the display control unit 129should control the display unit 150 such that the information indicatingthe completion of the calibration execution is displayed by the displayunit 150.

FIG. 7 is a diagram showing an example of the displayed informationindicating the completion of the calibration execution. As shown in FIG.7, in a case where the calibration execution unit 128 has completed theexecution of the calibration, the display control unit 129 may controldisplay of text data N1 “calibration completed” and an image N2indicating the completion of the calibration execution as an example ofinformation indicating the completion of the calibration execution. Notethat, although the display of both the text data N1 and the image N2 iscontrolled in the example shown in FIG. 7, the display of any one of thetext data N1 or the image N2 may be controlled.

Next, an operation example of the calibration according to the presentembodiment will be described. FIG. 8 is a flowchart showing theoperation example of the calibration according to the presentembodiment. As shown in FIG. 8, the motion acquisition unit 121 acquiresthe visual field motion of the user, and the object recognition unit 122determines whether or not the visual field motion of the user acquiredby the motion acquisition unit 121 is less than the predetermined motion(S11).

In a case where the visual field motion of the user acquired by themotion acquisition unit 121 is not less than the predetermined motion(“No” in S11), the object recognition unit 122 shifts the operation toS11. On the other hand, in a case where the visual field motion of theuser acquired by the motion acquisition unit 121 is less than thepredetermined motion (“Yes” in S11), the object recognition unit 122recognizes one or a plurality of objects from the visual field of theuser (S12).

Subsequently, the visual attraction degree calculation unit 123calculates the visual attraction degrees of the group of the objectsrecognized by the object recognition unit 122. Then, the target objectdetection unit 124 attempts to detect, from the visual field of theuser, a target object which has a visual attraction degree higher thanthe first threshold and a size in the predetermined direction smallerthan the predetermined size. In a case where such a target object is notdetected (“No” in S13), the target object detection unit 124 shifts theoperation to S13. On the other hand, in a case where such a targetobject has been detected (“Yes” in S13), the target object detectionunit 124 shifts the operation to S14.

Subsequently, the prediction information generation unit 125 generatesthe line-of-sight motion prediction information of the user on the basisof the visual attraction degrees of the group of the objects calculatedby the visual attraction degree calculation unit 123 (S14). Then, theline-of-sight information acquisition unit 127 acquires the userline-of-sight information, and the calibration execution unit 128acquires the line-of-sight information acquired in chronological orderby the line-of-sight information acquisition unit 127 and acquires, asthe actual line-of-sight motion measurement information, the informationin which these pieces of line-of-sight information are arranged in theacquisition order (S15).

Then, in a case where the matching degree between the line-of-sightmotion prediction information and the actual line-of-sight motionmeasurement information does not exceed the predetermined referencevalue (“No” in S16), it is considered that the user moves theline-of-sight on the basis of a criterion different from the visualattraction degree. Thus, the calibration execution unit 128 does notexecute the calibration in this case and ends the operation. On theother hand, in a case where the matching degree between theline-of-sight motion prediction information and the actual line-of-sightmotion measurement information exceeds the predetermined reference value(“Yes” in S16), it is considered that the user moves the line-of-sightin the descending order of the visual attraction degrees. Thus, thecalibration execution unit 128 executes the calibration in this case(S17) and ends the operation.

The operation example of the calibration according to the presentembodiment has been described above.

1.3.2. Recalibration

As described above, the calibration according to the present embodimentis executed. However, even in a case where the calibration is executedas described above, there is a possibility that a gap occurs between thedetection result of the position that the line-of-sight meets and theactual position that the line-of-sight meets as time passes. Forexample, there is a possibility that such a gap occurs due to a way thatthe information processing apparatus 10 is worn on the head of the user,the physical movement of the user wearing the information processingapparatus 10, and the like.

Accordingly, even in a case where the calibration is executed once asdescribed above, the recalibration should be executed at a certaintiming. Such recalibration will be described hereinafter. Note that, inthe following description, the calibration described as therecalibration does not have to be executed as the second and subsequentcalibrations and may be executed as the first calibration.

FIG. 9 is a diagram showing a first example of recalibration execution.Referring to FIG. 9, similar to the example shown in FIG. 5, a realspace R1 exists in the visual field of a user U1, and a group of objects(including buildings B1 to B3) exists in the real space R1. Herein,similar to the example shown in FIG. 5, the visual attraction degreecalculation unit 123 calculates, on the basis of the visual fieldanalysis image captured by the visual field analysis imaging unit iii,the visual attraction degrees of the group of the objects recognized bythe object recognition unit 122. Herein, it is considered that thevisual attraction degree of a region where no building exists (emptyregion) is calculated to be low. For example, presumed is a case wherethe visual attraction degree calculation unit 123 calculates the visualattraction degree of the region where no building exists to be lowerthan a fourth threshold.

Referring to FIG. 9, the line-of-sight of the user U1 meets regionswhere no building exists over a predetermined period of time. Morespecifically, the user U1 moves the line-of-sight over a predeterminedperiod of time in the regions where no building exists (the positionsthat the line-of-sight meets changes to a position Q31, a position Q32and a position Q33 in this order). In a case where the line-of-sightgoes to the regions with visual attraction degrees lower than the fourththreshold over a predetermined period of time in this way, it isconsidered that a gap is highly likely to occur between the detectionresult of the position that the line-of-sight meets and the actualposition that the line-of-sight meets.

Therefore, the calibration execution unit 128 should execute therecalibration in a case where the line-of-sight goes to the regions withvisual attraction degrees lower than the fourth threshold over apredetermined period of time Note that FIG. 9 has shown an example inwhich the recalibration is executed in a case where the user U1 hasmoved the line-of-sight over a predetermined period of time in theregions with the visual attraction degrees lower than the fourththreshold. However, the recalibration may also be executed in a casewhere the user U1 retains the line-of-sight over a predetermined periodof time in a region with a visual attraction degree lower than thefourth threshold.

The example, in which the recalibration is executed in a case where theline-of-sight goes to the regions with visual attraction degrees lowerthan the fourth threshold over the predetermined period of time, hasbeen described above. The timing at which the recalibration is executedis not limited to this example. For example, the calibration executionunit 128 may execute the calibration in a case where the state of themanipulation by the line-of-sight of the user is a predetermined state(e.g., in a case where the manipulation by the line-of-sight of the useris assumed to be not performed according to the desire of the user).

Herein, the manipulation by the line-of-sight of the user is notparticularly limited. Hereinafter, mainly presumed is a case where thedisplay control unit 129 controls display of a manipulation targetobject manipulatable by the user. The manipulation target object is notparticularly limited either. Hereinafter, mainly presumed is a casewhere the manipulation target object is a button, but the manipulationtarget object may be an icon or may be text data. In such a case,presumed is a case where selection of the manipulation target object isexecuted in a case where the line-of-sight of the user meets themanipulation target object over a predetermined period of time.

FIG. 10 is a diagram showing a second example of the recalibrationexecution. Referring to FIG. 10, similar to the example shown in FIG. 5,a real space R1 exists in the visual field of a user U1, and a group ofobjects (including buildings B1 to B3) exists in the real space R1.Furthermore, referring to FIG. 10, the display control unit 129 controlsthe display of a manipulation target object J1. In the example shown inFIG. 10, the manipulation target object J1 is a setting button, but themanipulation target object J1 is not limited to the setting button.

Herein, referring to FIG. 10, the line-of-sight of the user U1 actuallymeets the manipulation target object J1 in order to select themanipulation target object J1. However, the detection result of theposition that the line-of-sight of meets is a position Q41, which isdifferent from the actual position that the line-of-sight meets (theposition of the manipulation target object J1). In such a case, thecalibration execution unit 128 should execute the recalibration.

More specifically, the calibration execution unit 128 should execute therecalibration in a case where selection operation based on theline-of-sight for the manipulation target object J1 is not executedalthough the line-of-sight of the user acquired by the line-of-sightinformation acquisition unit 127 meets in a predetermined range over apredetermined period of time. Herein, the predetermined range is notparticularly limited. FIG. 10 shows a range C1 as the predeterminedrange. For example, the range C1 may be an inner region of a circle witha predetermined radius centered on the center of the locus of thedetection result of the position that the line-of-sight meets.

Furthermore, the calibration once executed may be disabled in a casewhere a certain condition is met. For example, in a case where the userhas done cancel operation, it is considered that a gap is highly likelyto occur between the detection result of the position that theline-of-sight meets and the actual position that the line-of-sightmeets. Accordingly, in a case where the user has done the canceloperation, the calibration execution unit 128 may disable thecalibration for the target object that the line-of-sight was meetingimmediately before the cancel operation. Note that the calibration to bedisabled is not limited to the calibration for the target object thatthe line-of-sight was meeting immediately before the cancel operation.For example, in a case where the user has done the cancel operation, thecalibration execution unit 128 may disable the calibration that has beenperformed until the cancel operation.

FIG. 11 is a diagram for describing an example of disabling thecalibration. Referring to FIG. 11, similar to the example shown in FIG.10, a real space R1 exists in the visual field of a user U1, and a groupof objects (including buildings B1 to B3) exists in the real space R1.Furthermore, referring to FIG. 11, similar to the example shown in FIG.10, the display control unit 129 controls the display of a manipulationtarget object J1.

In addition, in the example shown in FIG. 11, the display control unit129 controls the display of a manipulation target object J2. Althoughthe manipulation target object J2 is a help button, the manipulationtarget object J2 is not limited to the help button.

Herein, suppose that the line-of-sight of the user U1 actually meets themanipulation target object J1 in order to select the manipulation targetobject J1. However, presumed is a case where the detection result of theposition that the line-of-sight meets is the position of themanipulation target object J2, which is different from the actualposition that the line-of-sight meets (the position of the manipulationtarget object J1).

In such a case, the manipulation target object J2 is erroneouslyselected, and the display control unit 129 controls the display of ahelp screen. The help screen includes a manipulation target object J3.In the example shown in FIG. 11, the manipulation target object J3 is acancel button, but the manipulation target object J3 is not limited tothe cancel button.

Thereupon, the user U1 notices that the manipulation target object J2has been erroneously selected, and the line-of-sight meets themanipulation target object J3 as an example of the cancel operation.Note that mainly presumed herein is a case where the cancel operation ismanipulation in which the line-of-sight meets the manipulation targetobject J3, but the cancel operation is not limited to the manipulationin which the line-of-sight meets the manipulation target object J3. Forexample, the cancel operation may be an instruction by a predeterminedvoice, may be manipulation to press down the cancel button, or may be aninstruction by a predetermined gesture.

In FIG. 11, the position that the line-of-sight of the user U1 meets isindicated as a position Q51. At this time, the display control unit 129cancels the display of the help screen, and the calibration executionunit 128 disables the calibration for the target object that theline-of-sight was meeting immediately before the cancel operation. Forexample, in a case where the line-of-sight was meeting the target object(building B3) immediately before the cancel operation, the calibrationexecution unit 128 may disable the calibration executed on the basis ofthe position of the target object (building B3) and the line-of-sightmeeting the target object (building B3).

FIG. 12 is a diagram showing a third example of the recalibrationexecution. Referring to FIG. 12, similar to the example shown in FIG.11, a real space R1 exists in the visual field of a user U1, and a groupof objects (including buildings B1 to B3) exists in the real space R1.Furthermore, referring to FIG. 12, similar to the example shown in FIG.11, the display control unit 129 controls the display of a manipulationtarget object J1 and a manipulation target object J2.

Herein, suppose that the line-of-sight of the user U1 actually meets themanipulation target object J1 in order to select the manipulation targetobject J1. However, presumed is a case where the detection result of theposition that the line-of-sight meets is the position of themanipulation target object J2, which is different from the actualposition that the line-of-sight meets (the position of the manipulationtarget object J1).

In such a case, the manipulation target object J2 is erroneouslyselected, and the display control unit 129 controls the display of ahelp screen. The help screen includes a manipulation target object J3.In the example shown in FIG. 12, the manipulation target object J3 is acancel button, but the manipulation target object J3 is not limited tothe cancel button.

Thereupon, the user U1 notices that the manipulation target object J2has been erroneously selected, and the line-of-sight meets themanipulation target object J3 as an example of the cancel operation.Note that mainly presumed herein is a case where the cancel operation ismanipulation in which the line-of-sight meets the manipulation targetobject J3, but the cancel operation is not limited to the manipulationin which the line-of-sight meets the manipulation target object J3.

In FIG. 12, the position that the line-of-sight of the user U1 meets isindicated as a position Q52. At this time, the display control unit 129cancels the display of the help screen. Thereafter, the line-of-sight ofthe user U1 attempts to meet the manipulation target object J1 in orderto select the manipulation target object J1. However, the situation, inwhich the detection result of the position that the line-of-sight ofmeets is the position of the manipulation target object J2 which isdifferent from the actual position that the line-of-sight meets (theposition of the manipulation target object J1), may possibly occuragain.

Accordingly, the calibration execution unit 128 should execute therecalibration in a case where the same manipulation has been performedmore than a predetermined number of times within a predetermined periodof time. Specifically, the calibration execution unit 128 may executethe recalibration in a case where the selection operation based on theline-of-sight for the manipulation target object J2 has been performedmore than the predetermined number of times within the predeterminedperiod of time. Note that the predetermined number of times is notparticularly limited. For example, the predetermined number of times maybe any number as long as the number is two or more.

Next, an operation example of the recalibration according to the presentembodiment will be described. FIG. 13 is a flowchart showing theoperation example of the recalibration according to the presentembodiment. As shown in FIG. 13, in a case where the user has done thecancel operation (“Yes” in S21) the calibration execution unit 128disables the calibration for the target object that the line-of-sightwas meeting immediately before the cancel operation (S22) and shifts theoperation to S23. On the other hand, in a case where the user does notdo the cancel operation (“No” in S21), the calibration execution unit128 shifts the operation to S23.

Subsequently, on the basis of the visual attraction degrees of the groupof the objects calculated by the visual attraction degree calculationunit 123, the calibration execution unit 128 detects a region with avisual attraction degree lower than the fourth threshold as a low visualattraction degree region. In a case where the line-of-sight meets thislow visual attraction degree region (“Yes” in S23), the calibrationexecution unit 128 shifts the operation to S26. On the other hand, in acase where the line-of-sight does not meet this low visual attractiondegree region (“No” in S23), the calibration execution unit 128 shiftsthe operation to S24.

Subsequently, the calibration execution unit 128 determines whether ornot the selection operation based on the line-of-sight of the user isnot executed (S24). More specifically, the calibration execution unit128 determines whether or not the selection operation based on theline-of-sight for the manipulation target object is not executedalthough the line-of-sight of the user acquired by the line-of-sightinformation acquisition unit 127 meets in the predetermined range overthe predetermined period of time. In a case where the selectionoperation based on the line-of-sight of the user is not executed (“Yes”in S24), the calibration execution unit 128 shifts the operation to S26.On the other hand, in a case where the selection operation based on theline-of-sight of the user is executed (“No” in S24), the calibrationexecution unit 128 shifts the operation to S25.

Subsequently, the calibration execution unit 128 determines whether ornot the same manipulation is repeated (S25). More specifically, thecalibration execution unit 128 determines whether or not the samemanipulation has been performed more than the predetermined number oftimes within the predetermined period of time. In a case where the samemanipulation is repeated (“Yes” in S25), the calibration execution unit128 executes the recalibration (S26) and ends the operation. On theother hand, in a case where the same operation is not repeated (“No” inS25), the calibration execution unit 128 ends the operation.

The operation example of the recalibration according to the presentembodiment has been described above.

1.3.3. Various Modification Examples

As described above, the calibration according to the present embodimentis executed. Furthermore, as described above, the recalibrationaccording to the present embodiment may be executed. Herein, thetechnique of detecting a target object (e.g., the building B3) utilizedfor the calibration (hereinafter, also referred to as “the techniquealready described”) has already been described. However, a techniqueaccording to a modification example may be utilized to detect a targetobject utilized for the calibration. Hereinafter, the techniqueaccording to this modification example will be described.

Note that the detection of a target object by the technique according tothe modification example may be performed at any timing. As one example,the detection of a target object by the technique according to themodification example may be performed in a case where the target objectis not detected by the technique already described. Alternatively, asanother example, the detection of a target object by the techniqueaccording to the modification example may be performed before thedetection of the target object by the technique already described isattempted.

FIG. 14 is a diagram for describing a first example of the techniqueaccording to the modification example. As shown in FIG. 14, in thevisual field of a user U1, a desk B4 and a writing instrument B5 existas an example of one or a plurality of objects existing in a real spaceR2. Then, the user U1 tries to take the writing instrument B5 placed onthe desk B4. Therefore, a position Q61 that the line-of-sight of theuser U1 meets exists at the writing instrument B5. Thus, the user U1 ishighly likely to be looking at the writing instrument B5 that the userU1 is trying to take.

Similarly, in a case where the gesture of the user U1 is a predeterminedgesture, it is considered that the line-of-sight of the user U1 highlylikely meets the object existing at the position corresponding to theposition at which the gesture has been made. Specifically, an objectrecognition unit 122 recognizes the gesture of the user U1 from a visualfield analysis image captured by a visual field analysis imaging unit111. Then, a target object detection unit 124 determines whether or notthe gesture recognized by the object recognition unit 122 is thepredetermined gesture.

Then, in a case where the gesture recognized by the object recognitionunit 122 is the predetermined gesture, the target object detection unit124 detects, as a target object, an object existing at a positioncorresponding to the position at which the gesture has been made. Morespecifically, in a case where the gesture recognized by the objectrecognition unit 122 is the predetermined gesture, the target objectdetection unit 124 identifies the correspondence between the position atwhich the gesture has been made and the position of the object for thegesture. Such correspondence may be registered in advance or may beestimated by the target object detection unit 124 on the basis ofpredetermined algorithm. Then, the target object detection unit 124detects, as the target object, the object for the gesture on the basisof the correspondence.

Note that the predetermined gesture may include at least one of agesture of holding an object or a gesture of pointing at an object. FIG.14 shows the case where the gesture of the user U1 is the gesture ofholding an object. In this case, the target object detection unit 124 isonly required to identify the correspondence between the positions of aplurality of fingers of the user U1 and the positions of the writinginstrument B5 held by the plurality of fingers and detect, as the targetobject, the writing instrument B5 held by the plurality of fingers onthe basis of the correspondence. A calibration execution unit 128executes the calibration on the basis of the positions of the targetobject and the line-of-sight of the user U1.

FIG. 15 is a diagram for describing a second example of the techniqueaccording to the modification example. As shown in FIG. 15, in thevisual field of a user U1, a desk B4 exists as an example of one or aplurality of objects existing in a real space R2. Furthermore, a displaycontrol unit 129 controls a display unit 150 so that a display object D1is displayed. Herein, presumed is a case where the position of thedisplay object D1 is moving in order to increase the possibility thatthe line-of-sight of the user U1 meets the display object D1, but theposition of the display object D1 does not have to change.

When the display object D1 is displayed in this manner, it is consideredthat the user U1 is highly likely to look at the display object D1.Therefore, a position Q71 that the line-of-sight of the user U1 meetsexists in the display object D1. Accordingly, a target object detectionunit 124 is only required to detect the display object D1 as a targetobject. A calibration execution unit 128 executes the calibration on thebasis of the position of the target object and user line-of-sightinformation of the user U1.

FIG. 16 is a diagram for describing a third example of the techniqueaccording to the modification example. As shown in FIG. 16, in thevisual field of the user U1, a desk B4 exists as one or a plurality ofobjects existing in a real space R2. Furthermore, a display control unit129 controls a display unit 150 so that a manipulation target object J1is displayed. Also in the example shown in FIG. 16, similar to theexample shown in FIG. 10, the manipulation target object J1 is a settingbutton, but the manipulation target object J1 is not limited to thesetting button.

When the manipulation target object J1 is displayed in this manner, itis considered that the user U1 is highly likely to look at themanipulation target object J1. Therefore, a position Q81 that theline-of-sight of the user U1 meets exists in the manipulation targetobject J1. Accordingly, a target object detection unit 124 is onlyrequired to detect the manipulation target object J1 as a target object.A calibration execution unit 128 executes the calibration on the basisof the positions of the target object and the line-of-sight of the userU1.

2. Hardware Configuration Example

Next, a hardware configuration example of the information processingapparatus 10 according to the embodiment of the present disclosure willbe described with reference to FIG. 17. FIG. 17 is a block diagramshowing the hardware configuration example of the information processingapparatus 10 according to the embodiments of the present disclosure.

As shown in FIG. 17, the information processing apparatus 10 includes acentral processing unit (CPU) 901, a read only memory (ROM) 903 and arandom access memory (RAM) 905. Furthermore, the information processingapparatus 10 may include a host bus 907, a bridge 909, an external bus911, an interface 913, an input apparatus 915, an output apparatus 917,a storage apparatus 919, a drive 921, a connection port 923 and acommunication apparatus 925. Moreover, the information processingapparatus 10 may include an imaging apparatus 933 and a sensor 935 asnecessary. Instead of or in addition to the CPU 901, the informationprocessing apparatus 10 may include a processing circuit called adigital signal processor (DSP) or an application specific integratedcircuit (ASIC).

The CPU 901 functions as an arithmetic processing unit and a controlunit and controls the entire or partial operation in the informationprocessing apparatus 10 according to various programs recorded on theROM 903, the RAM 905, the storage apparatus 919, or a removablerecording medium 927. The ROM 903 stores programs, arithmetic operationparameters, and the like, which are used by the CPU 901. The RAM 905temporarily stores programs used in the execution of the CPU 901,parameters that change as appropriate in the execution, and the like.The CPU 901, the ROM 903 and the RAM 905 are connected to each other bythe host bus 907 constituted by an internal bus such as a CPU bus.Moreover, the host bus 907 is connected to the external bus 911, such asa peripheral component interconnect/interface (PCI) bus, via the bridge909.

The input apparatus 915 is an apparatus manipulated by the user, suchas, for example, a mouse, a keyboard, a touch panel, a button, a switchand a lever. The input apparatus 915 may include a microphone thatdetects the voice of the user. The input apparatus 915 may be, forexample, a remote control apparatus utilizing infrared rays or otherelectric waves or may be an external connection device 929, such as amobile phone, supporting the manipulation of the information processingapparatus 10. The input apparatus 915 includes an input control circuitthat generates an input signal on the basis of the information input bythe user and outputs the input signal to the CPU 901. The usermanipulates this input apparatus 915 to input various data into theinformation processing apparatus 10 and instruct the informationprocessing apparatus 10 to perform processing operation. Furthermore,the imaging apparatus 933 as described later can also function as theinput apparatus by imaging the motion of the hand of the user, thefinger of the user, or the like. At this time, a pointing position maybe decided according to the motion of the hand or the direction of thefinger.

The output apparatus 917 is constituted by an apparatus capable ofvisually or auditorily notifying the user of the acquired information.For example, the output apparatus 917 can be a display apparatus, suchas a liquid crystal display (LCD), a plasma display panel (PDP), anorganic electro-luminescence (EL) display or a projector, a hologramdisplay apparatus, an audio output apparatus such as a speaker or aheadphone, as well as a printer apparatus, or the like. The outputapparatus 917 outputs the result obtained by the processing of theinformation processing apparatus 10 as a picture such as a text or animage or outputs the result as sound such as voice or audio.Furthermore, the output apparatus 917 may include a light forbrightening the surroundings.

The storage apparatus 919 is a data storage apparatus constituted as oneexample of a storage unit of the information processing apparatus 10.The storage apparatus 919 is constituted by, for example, a magneticstorage unit device such as a hard disk drive (HDD), a semiconductorstorage device, an optical storage device, a magneto-optical storagedevice, or the like. This storage apparatus 919 stores programs executedby the CPU 901 and various data as well as various data acquired fromthe outside, and the like.

The drive 921 is a reader/writer for the removable recording medium 927such as a magnetic disk, an optical disk, a magneto-optical disk, or asemiconductor memory and is built in the information processingapparatus 10 or externally attached thereto. The drive 921 reads out theinformation recorded on the attached removable recording medium 927 andoutputs the information to the RAM 905. Furthermore, the drive 921writes a record in the attached removable recording medium 927.

The connection port 923 is a port for directly connecting the device tothe information processing apparatus 10. The connection port 923 can be,for example, a universal serial bus (USB) port, an IEEE 1394 port, asmall computer system interface (SCSI) port, or the like. Furthermore,the connection port 923 may be an RS-232C port, an optical audioterminal, a high-definition multimedia interface (HDMI (registeredtrademark)) port, or the like. By connecting the external connectiondevice 929 to the connection port 923, various data can be exchangedbetween the information processing apparatus 10 and the externalconnection device 929.

The communication apparatus 925 is, for example, a communicationinterface constituted by a communication device or the like forconnecting to a communication network 931. The communication apparatus925 can be, for example, a communication card for a wired or wirelesslocal area network (LAN), Bluetooth (registered trademark), or awireless USB (WUSB). Furthermore, the communication apparatus 925 may bea router for optical communication, a router for an asymmetric digitalsubscriber line (ADSL), or a modem for various kinds of communication.For example, the communication apparatus 925 transmits and receivessignals and the like to and from the Internet and other communicationdevices by using a predetermined protocol such as TCP/IP. Moreover, thecommunication network 931 connected to the communication apparatus 925is a network connected by wire or wireless, and is, for example, theInternet, a home LAN, infrared communication, radio wave communication,satellite communication, or the like.

The imaging apparatus 933 is an apparatus that images a real space byusing various members, such as an imaging element (e.g., a chargedcoupled device (CCD) or a complementary metal oxide semiconductor(CMOS)) and a lens for controlling the formation of a subject image onthe imaging element, and generates a captured image. The imagingapparatus 933 may capture a still image or may capture a moving image.

The sensor 935 is, for example, various sensors such as a distancemeasuring sensor, an acceleration sensor, a gyro sensor, a geomagneticsensor, an optical sensor and a sound sensor. The sensor 935 acquires,for example, information regarding the state of the informationprocessing apparatus 10 itself, such as the posture of the casing of theinformation processing apparatus 10, and information regarding thesurrounding environment of the information processing apparatus 10, suchas brightness and noise around the information processing apparatus 10.Furthermore, the sensor 935 may include a GPS sensor that receives aglobal positioning system (GPS) signal and measures the latitude,longitude and altitude of the apparatus.

3. Conclusion

As described above, according to the embodiments of the presentdisclosure, provided is an information processing apparatus 10including: a line-of-sight information acquisition unit 127 configuredto acquire user line-of-sight information; and a calibration executionunit 128 configured to execute calibration on the basis of a position ofa target object in a visual field and the line-of-sight information in acase where the target object, which has a visual attraction degreehigher than a first threshold and a size in a predetermined directionsmaller than a predetermined size, has been detected from the visualfield of the user. According to such configuration, it is possible toenable the calibration of the line-of-sight to be executed morenaturally.

Although the preferred embodiments of the present disclosure have beendescribed in detail with reference to the accompanying drawings, thetechnical scope of the present disclosure is not limited to theseexamples. It is obvious that a person ordinarily skilled in the art ofthe present disclosure can arrive at various changing examples ormodification examples within the scope of the technical idea describedin the claims, and it is to be understood that these are also within thetechnical scope of the present disclosure as a matter of course.

For example, the example, in which the visual attraction degreecalculation unit 123 calculates the visual attraction degrees of thegroup of the objects when the target object detection unit 124 detects,on the basis of the visual attraction degrees of the group of theobjects, the target object utilized for the calibration, has beendescribed above. However, the visual attraction degree calculation unit123 may calculate in advance the visual attraction degrees of the groupof the objects in a case, for example, where an image containing thetarget object utilized for the calibration can be acquired in advance.

Furthermore, the target object detected by the target object detectionunit 124 may be different depending on the user. For example, the targetobject detected by the target object detection unit 124 may be detectedon the basis of user preference information. Moreover, the target objectdetected by the target object detection unit 124 may be detected on thebasis of the line-of-sight of another user. At this time, another usermay be a user whose preference information is similar to that of theuser.

Further, the information indicating the object that the line-of-sight ofthe user meets may be accumulated as a past history. Accordingly, in acase where the object indicated by the past history of the user comesinto the visual field of the user again, the target object detectionunit 124 may determine that the object is highly likely to be looked atby the user and, therefore, preferentially detect the object as thetarget object.

Furthermore, in order for the target object detection unit 124 to drawmore visual attention of the user to the target object, the displaycontrol unit 129 may control a predetermined object to be displayed atthe position for the target object by the target object detection unit124. For example, the object whose display is controlled by the displaycontrol unit 129 may be a frame surrounding the periphery of the targetobject, may be coloring on the target object itself or the periphery ofthe target object, or flickering of the target object itself or theperiphery of the target object.

Moreover, it is also possible to create a program for causing thehardware such as CPU, ROM and RAM built in the computer to exhibitfunctions equivalent to the functions of the control unit 120 describedabove. Further, a computer-readable recording medium on which theprogram is recorded can also be provided.

For example, as long as the operation of the information processingapparatus 10 described above is realized, the position of eachconstituent is not particularly limited. Part of the processing of eachunit in the information processing apparatus 10 may be performed by aserver apparatus (not shown). As one specific example, each partial orentire block of the control unit 120 in the information processingapparatus 10 may exist in the server apparatus (not shown) or the like.For example, the partial or entire functions of the object recognitionunit 122, the visual attraction degree calculation unit 123, the targetobject detection unit 124, the prediction information generation unit125 and the gesture recognition unit 126 in the information processingapparatus 10 may exist in the server apparatus (not shown) or the like.

Furthermore, the effects described in this specification are merelyillustrative or exemplary, and not limited. That is, the technologyaccording to the present disclosure can exert other effects obvious tothose skilled in the art from the description of this specification,together with the above-described effects or instead of theabove-described effects.

Note that the following configurations also within the technical scopeof the present disclosure.

(1)

An information processing apparatus including:

a line-of-sight information acquisition unit configured to acquireline-of-sight information of a user; and

a calibration execution unit configured to execute calibration on thebasis of a position of a target object in a visual field and theline-of-sight information, the target object having a visual attractiondegree higher than a first threshold and a size in a predetermineddirection smaller than a predetermined size, in a case where the targetobject has been detected from the visual field of the user.

(2)

The information processing apparatus according to (1), further including

a target object detection unit configured to detect the target object.

(3)

The information processing apparatus according to (2), in which

the target object detection unit is configured to detect the targetobject in a case where motion of the visual field is less thanpredetermined motion.

(4)

The information processing apparatus according to (2) or (3), in which

the target object detection unit is configured not to detect one objectas the target object in a case where another object is located at adistance from the one object shorter than a predetermined distance, theanother object having a visual attraction degree higher than a secondthreshold.

(5)

The information processing apparatus according to any one of (2) to (4),in which

the target object detection unit is configured to detect, as the targetobject, an object which exists at a position corresponding to a positionat which a gesture has been made, in a case where the gesture of theuser is a predetermined gesture.

(6)

The information processing apparatus according to (5), in which

the predetermined gesture includes at least one of a gesture of holdingthe object or a gesture of pointing at the object.

(7)

The information processing apparatus according to any one of (1) to (6),in which

the calibration execution unit is configured to control whether or notto execute the calibration on the basis of prediction informationregarding line-of-sight motion.

(8)

The information processing apparatus according to (7), further including

a prediction information generation unit configured to generate, as theprediction information regarding the line-of-sight motion, informationin which respective positions of a plurality of objects are arranged indescending order of visual attraction degrees, in a case where theplurality of the objects has been detected from the visual field of theuser, the plurality of the objects having visual attraction degreeshigher than a third threshold.

(9)

The information processing apparatus according to (7) or (8), in which

the calibration execution unit is configured not to execute thecalibration in a case where a matching degree between the predictioninformation regarding the line-of-sight motion and actual measurementinformation regarding the line-of-sight motion does not exceed apredetermined reference value.

(10)

The information processing apparatus according to (9), in which

the calibration execution unit is configured to execute the calibrationin a case where the matching degree exceeds the reference value.

(11)

The information processing apparatus according to any one of (1) to(10), in which

the calibration execution unit is configured to execute the calibrationfor a direction substantially parallel to the predetermined direction ina case where the target object has been detected.

(12)

The information processing apparatus according to any one of (1) to(11), in which

the calibration execution unit is configured to execute the calibrationin a case where a line-of-sight exists, over a predetermined period oftime, in a region with the visual attraction degree lower than a fourththreshold.

(13)

The information processing apparatus according to any one of (1) to(12), in which

the calibration execution unit is configured to execute the calibrationin a case where a state of manipulation by the line-of-sight of the useris a predetermined state.

(14)

The information processing apparatus according to (13), in which

the calibration execution unit is configured to execute the calibrationin a case where selection operation based on the line-of-sight is notexecuted although the line-of-sight meets in a predetermined range overa predetermined period of time.

(15)

The information processing apparatus according to (13), in which

the calibration execution unit is configured to execute the calibrationin a case where same manipulation has been performed more than apredetermined number of times within a predetermined period of time.

(16)

The information processing apparatus according to any one of (1) to(15), in which

the calibration execution unit is configured to execute the calibrationon the basis of a position of a predetermined display object and theline-of-sight information, in a case where the target object is notdetected from the visual field of the user.

(17)

The information processing apparatus according to any one of (1) to(15), in which

the calibration execution unit is configured to execute the calibrationon the basis of a position of a manipulation target object manipulatableby the user and the line-of-sight information, in a case where thetarget object is not detected from the visual field of the user.

(18)

The information processing apparatus according to any one of (1) to(17), in which

the calibration execution unit is configured to disable calibration fora target object that a line-of-sight was meeting immediately beforecancel operation, in a case where the user has done the canceloperation.

(19)

An information processing method including:

acquiring line-of-sight information of a user; and

executing calibration by a processor on the basis of a position of atarget object in a visual field and the line-of-sight information, thetarget object having a visual attraction degree higher than a firstthreshold and a size in a predetermined direction smaller than apredetermined size, in a case where the target object has been detectedfrom the visual field of the user.

(20)

A program for causing a computer to function as an informationprocessing apparatus, the information processing apparatus including:

a line-of-sight information acquisition unit configured to acquireline-of-sight information of a user; and

a calibration execution unit configured to execute calibration on thebasis of a position of a target object in a visual field and theline-of-sight information, the target object having a visual attractiondegree higher than a first threshold and a size in a predetermineddirection smaller than a predetermined size, in a case where the targetobject has been detected from the visual field of the user.

REFERENCE SIGNS LIST

-   10 Information processing apparatus-   111 Visual field analysis imaging unit-   112 Line-of-sight detection imaging unit-   113 Sensor unit-   120 Control unit-   121 Acquisition unit-   122 Object recognition unit-   123 Visual attraction degree calculation unit-   124 Target object detection unit-   125 Prediction information generation unit-   126 Gesture recognition unit-   127 Line-of-sight information acquisition unit-   128 Calibration execution unit-   129 Display control unit-   130 Storage unit-   140 Communication unit-   150 Display unit

The invention claimed is:
 1. An information processing apparatus,comprising: a line-of-sight information acquisition unit configured toacquire line-of-sight information of a user; and a calibration executionunit configured to execute calibration of the line-of-sight informationbased on a position of a target object in a visual field and theline-of-sight information, wherein the target object has a visualattraction degree higher than a first threshold degree, the visualattraction degree is a numerical value indicating how much visualattention is paid by the user, the target object has a size in aspecific direction smaller than a specific size, in a case where thetarget object has been detected from the visual field of the user, andthe calibration is executed for a direction parallel to the specificdirection.
 2. The information processing apparatus according to claim 1,further comprising a target object detection unit configured to detectthe target object.
 3. The information processing apparatus according toclaim 2, wherein the target object detection unit is configured todetect the target object in a case where motion of the visual field isless than a specific motion.
 4. The information processing apparatusaccording to claim 2, wherein the target object detection unit isconfigured not to detect one object as the target object in a case whereanother object is located at a distance from the one object shorter thana predetermined distance, the another object having a visual attractiondegree higher than a second threshold degree.
 5. The informationprocessing apparatus according to claim 2, wherein the target objectdetection unit is configured to detect, as the target object, an objectwhich exists at the position corresponding to a position at which agesture has been made, in a case where the gesture of the user is aspecific gesture.
 6. The information processing apparatus according toclaim 5, wherein the specific gesture includes at least one of a gestureof holding the object or a gesture of pointing at the object.
 7. Theinformation processing apparatus according to claim 1, wherein thecalibration execution unit is further configured to control whether toexecute the calibration based on prediction information regardingline-of-sight motion.
 8. The information processing apparatus accordingto claim 7, further comprising a prediction information generation unitconfigured to generate, as the prediction information regarding theline-of-sight motion, information in which respective positions of aplurality of objects are in a descending order of visual attractiondegrees of the plurality of objects, in a case where the plurality ofthe objects are detected from the visual field of the user, theplurality of the objects having the visual attraction degrees higherthan a third threshold degree.
 9. The information processing apparatusaccording to claim 7, wherein the calibration execution unit is furtherconfigured not to execute the calibration in a case where a matchingdegree between the prediction information regarding the line-of-sightmotion and actual measurement information regarding the line-of-sightmotion does not exceed a reference value.
 10. The information processingapparatus according to claim 9, wherein the calibration execution unitis further configured to execute the calibration in a case where thematching degree exceeds the reference value.
 11. The informationprocessing apparatus according to claim 1, wherein the calibrationexecution unit is further configured to execute the calibration in acase where a line-of-sight exists, over a period of time, in a regionwith the visual attraction degree lower than a fourth threshold degree.12. The information processing apparatus according to claim 1, whereinthe calibration execution unit is further configured to execute thecalibration in a case where a state of manipulation by a line-of-sightof the user is a specific state.
 13. The information processingapparatus according to claim 12, wherein the calibration execution unitis configured to execute the calibration in a case where a selectionoperation based on the line-of-sight is not executed although theline-of-sight meets in a range over a period of time.
 14. Theinformation processing apparatus according to claim 12, wherein thecalibration execution unit is configured to execute the calibration in acase where same manipulation is performed more than a number of timeswithin a period of time.
 15. The information processing apparatusaccording to claim 1, wherein the calibration execution unit is furtherconfigured to execute the calibration based on a position of a displayobject and the line-of-sight information, in a case where the targetobject is not detected from the visual field of the user.
 16. Theinformation processing apparatus according to claim 1, wherein thecalibration execution unit is further configured to execute thecalibration based on a position of a manipulation target objectmanipulatable by the user and the line-of-sight information, in a casewhere the target object is not detected from the visual field of theuser.
 17. The information processing apparatus according to claim 1,wherein the calibration execution unit is further configured to disablethe calibration for the target object where a line-of-sight was meetingimmediately before a cancel operation, based on the cancel operation bythe user.
 18. An information processing method, comprising: in aninformation processing apparatus: acquiring line-of-sight information ofa user; and executing calibration of the line-of-sight information basedon a position of a target object in a visual field and the line-of-sightinformation, wherein the target object has a visual attraction degreehigher than a first threshold degree, the visual attraction degree is anumerical value indicating how much visual attention is paid by theuser, the target object has a size in a specific direction smaller thana specific size, in a case where the target object has been detectedfrom the visual field of the user, and the calibration is executed for adirection parallel to the specific direction.
 19. A non-transitorycomputer-readable medium having stored thereon, computer-executableinstructions which, when executed by a computer cause the computer toexecute operations, the operations comprising: acquiring line-of-sightinformation of a user; and executing calibration of the line-of-sightinformation based on a position of a target object in a visual field andthe line-of-sight information, wherein the target object has a visualattraction degree higher than a first threshold degree, the visualattraction degree is a numerical value indicating how much visualattention is paid by the user, the target object has a size in aspecific direction smaller than a specific size, in a case where thetarget object has been detected from the visual field of the user, andthe calibration is executed for a direction parallel to the specificdirection.